Camera chip, camera and method for image recording

ABSTRACT

The invention relates to a camera chip (C) for image acquisition. It is characterized in that pixel groups (P 1 , P 2 , . . . ) may be exposed at different times with the aid of shutter signals (S 1 , S 2 , . . . ).

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a 371 U.S. National Stage of InternationalApplication No. PCT/EP2006/010790, filed Nov. 10, 2006. This applicationclaims priority to German Patent Applications No. 10 2005 054 465.7,filed Nov. 10, 2005, No. 10 2006 033 391.8, filed Jul. 13, 2006, and No.10 2006 041 932.4, filed Sep. 7, 2006. The disclosures of the aboveapplications are herein expressly incorporated by reference.

FIELD

The invention relates to a camera chip for image acquisition, anelectronic camera, a method for optical shape capture, a method foracquiring a number of images, a method for acquiring a number of imagesusing multiple cameras, as well as a method for acquiring a number ofimages.

BACKGROUND

There are already a large number of measurement principles today bymeans of which it is possible to capture the three-dimensional shape ofobjects. Examples include the principles of triangulation (laser lightsection method, stripe projection, stereo method, photogrammetry, shapefrom shading), interferometric methods (laser interferometry, whitelight interferometry, holographic methods), and runtime methods(high-frequency modulation of the light source). All of these methodshave in common the fact that multiple camera images must be acquired inorder to produce one single 3-D image from them. In the case of most ofthese methods, it is not possible for these images to be acquired at thesame time. In contrast, conventional photographic 2-D images arecaptured with only one image. This applies in particular to imagecapture in industrial image processing as well.

The principal difficulty lies in the fact that the use of optical 3-Dsensors requires multiple camera captures by its nature. This lies inthe fact every 3-D sensor must determine three unknowns for each pointof the object to be measured:

-   -   the location of the test specimen point, referred to in the        following as “shape”    -   the local reflectivity of the test specimen, referred to in the        following as “texture” (black/white)    -   the local brightness of the ambient light at each point

As a rule, three equations are also necessary to determine threeunknowns; in terms of 3-D, these are the three camera images with thelocal brightness of the test specimen, with the three camera imagesbeing acquired using three different lighting situations. This is notnecessary in the case of the 2-D methods because here only the sum ofall influences, e.g., of shape, texture, and ambient light, is everreproduced in one image.

The number of unknowns is reduced to two unknowns insofar as it ispossible for all ambient light to be shaded off. In this case, only twoimages with two different lighting situations are necessary to detectthe 3-D shape of the test specimen.

Even in 3-D methods, there are approaches that allow all of thenecessary information to be attained using one single camera image. Onesuch method is spatial phase shifting. In its application, it is limitedto interferometry and stripe projection. In this method, the variouslighting situations are realized using an interference pattern or astripe pattern that has regions of different lighting intensities.Therefore, it is possible to detect three different lighting situationsat three neighboring picture elements, from which the three unknowns maythen be calculated. However, this method cannot be applied to the shapefrom shading method, in particular not for photometric stereo orphotometric deflectometry (see WO2004051186, DE102005013614) or forruntime methods because, in this case, an object is lit from differentdirections at different times and thus a simultaneous recording ofmultiple lighting situations is not possible.

However, for many applications, in particular for applications inindustrial image processing, it is important for all image informationto be recorded simultaneously or virtually simultaneously. Only in thismanner is it possible for test specimens in motion to be measured andanalyzed without blurring from movement. For this purpose, the exposuretime of a camera is reduced to a minimum or a flash is selected. Thecameras are equipped with a so-called electronic shutter in order tocontrol the exposure time. All picture elements (pixels) of the camerachip are connected simultaneously in a photosensitive manner for apredetermined time. Typical exposure times range from a few millisecondsto several microseconds. The process of reading the camera chip occursafter exposure. Here, exposure may occur much more quickly than reading,which normally takes several milliseconds. Therefore, an imageacquisition period is composed of exposure and reading the cameral chip.Here, the duration of the image acquisition period is determined by thereading time, which takes substantially longer than exposure. By virtueof the long image acquisition period, the refresh rate, i.e., the numberof images that may be recorded per second, is reduced. The refresh rateis therefore also determined by the reading time. Specially designed andexpensive high-speed cameras which, for example, are able to record afew thousand images per second, represent the exception. Thus, theoptical 3-D method has a decisive disadvantage. Instead of an exposuretime of a few microseconds, a series of, for example, four images (forexample, 20 ms each per image acquisition) requires an acquisition timeof 80 ms, over 1000 times that of the 2-D method.

SUMMARY

The object of the invention is therefore to create an optical 3-D sensorthat allows a significantly shorter acquisition time. This sensor isreferred to as a “single-shot 3-D sensor.”

The further object of the invention is to create a camera and method foroptical shape capture requiring only a very short exposure time.

In order to attain the objects mentioned above, a camera chip, a camera,and a method are created that are generally suitable for optical 3-Dprocesses and allow several images to be acquired within the shortesttime.

DRAWINGS

The invention is described in greater detail below with reference to thedrawings, which show:

FIG. 1 a a basic sketch of a camera chip with a number of pixel groups;

FIG. 1 b an exemplary progression of shutter signals over time;

FIGS. 2 a to d a basic sketch of photometric deflectometry with fourshutters and four illuminations;

FIG. 2 e a basic sketch of photometric stereo with four shutters andfour illuminations;

FIGS. 3 a to d a basic sketch of the stripe projection method with fourshutters and four illuminations;

FIG. 3 e a basic sketch of interferometry with four shutters and fourilluminations;

FIG. 4 a a schematic depiction of a partial region of a modified camerachip;

FIG. 4 b an exemplary progression of shutter signals over time;

FIG. 5 an exemplary embodiment of a camera chip C;

FIG. 6 a diagram to show control of shutter and illumination signalsover time;

FIG. 7 a modified exemplary embodiment of a camera chip C;

FIGS. 8 a to d a basic sketch of photometric deflectometry in the caseof linear movement by the test specimen with four illuminations and fourcamera chip lines;

FIG. 8 e a basic sketch of photometric deflectometry in the case ofrotational movement by the test specimen with four illuminations andfour camera chip lines;

FIG. 9 a upper diagram: progression over time of four active camera chiplines as a function of the location on the surface of the test specimen;lower diagram: corresponding active illuminations;

FIG. 9 b a schematic depiction of a memory area for storing read camerachip areas;

FIG. 10 a a modified schematic depiction for storing read camera chipareas in a memory area;

FIG. 10 b a modified diagram for the purpose of clarifying the temporaland spatial progression of the activation of camera chip lines inacquiring various images; lower diagram: corresponding activeilluminations;

FIG. 10 c a schematic depiction for the purpose of storing read camerachip areas in a memory area;

FIG. 11 a a basic sketch of photometric stereo in the case of linearmovement by the test specimen;

FIG. 11 b a basic sketch of photometric stereo in the case of rotationalmovement by the test specimen;

FIGS. 12 a to d a basic sketch of the stripe projection method in thecase of linear movement;

FIG. 12 e a basic sketch of the stripe projection method in the case ofrotational movement by the test specimen;

FIG. 13 a a basic sketch of interferometric evaluation in the case ofrotational movement by the test specimen;

FIG. 13 b a basic sketch of interferometric evaluation in the case oflinear movement by the test specimen;

FIG. 14 a a basic sketch of the white light interferometry method in thecase of linear movement by the test specimen;

FIG. 14 b a basic sketch of the white light interferometry method in thecase of rotational movement by the test specimen;

FIG. 15 an arrangement of two cameras in a vertical fashion over a planein which a test specimen has been arranged;

FIGS. 16 a to d a modified basic sketch of photometric deflectometry inthe case of rotational movement by the test specimen with fourilluminations and one line, and

FIG. 16 e a basic sketch of photometric deflectometry with a highlylight-sensitive line camera.

DETAILED DESCRIPTION

According to the invention, a specially designed camera chip C of anelectronic camera K is used for image acquisition, for example, usingCCD or CMOS technology.

FIG. 1 a shows a schematic section of a camera chip C with multiplepixels P. Here, individual pixels P are combined into pixel groups P1,P2, P3, and P4. These pixel groups P1 to P4 are activated via variouselectronic shutter signals S1, S2, S3, S4, i.e., they are connected in alight-sensitive manner. Here, precisely one shutter S1 to Sn is assignedto each pixel group P1 to Pn. Individual pixels P are arranged at adistance Δx and/or Δy from one another. This distance is preferablyselected to be very small in order to achieve full coverage by thedistribution of the pixels P, such that the yield for the camera chiparea is large.

In the following figures, only a few pixels P or pixel groups P1 to Pnof the camera chip C are shown. However, each respective pattern must beimagined as continuing in the horizontal and vertical directions untilthe desired number of pixels P, for example, two megapixels, has beenachieved. Shutters Sn are assigned to these pixels. An electronicshutter is understood to mean an illumination control that is able toconnect a pixel P for a certain time in a light-sensitive manner or in anon-light-sensitive manner. In this manner, it is possible forneighboring pixels P of various pixel groups P1 to Pn to be exposed withdifferent images in rapid succession over time. The camera chip C willnot be read until all pixel groups P1 to Pn have been exposed one afterthe other. Therefore, upon reading, there are as many images in partialregions of the camera chip C as pixel groups that were connected oneafter the other in a light-sensitive fashion by shutters. Because thereading process determines the image period, i.e., the sum of theexposure and reading times, and thus the refresh rate, i.e., the numberof images acquired per second, it is possible in this manner for severalimages to be acquired within the shortest period of time without aneeding to wait for a comparably protracted reading process after eachexposure process.

FIG. 1 b shows a progression over time of the various shutter signals,with t signifying the time axis. The shutter signals are shown asright-angle pulses over a certain time interval Δt. The light sourcesdesignated as illuminations must be synchronized with the shuttersignals, for example, illumination B1 with shutter S1, illumination B2with shutter S2, etc. For example, the shutter S1 is actively connectedfor 20 microseconds, then shutters S2, S3, S4 for 20 microseconds each.After only 80 microseconds, four images are present in four neighboringpixels that were captured at different times and with differentillumination conditions and that were acquired in accordance with aknown 3-D process, for example, shape from shading, and are then able tobe read from the camera chip. In particular, the photometric stereomethod and photometric deflectometry method are possible in anadvantageous fashion with four illuminations (see WO2004051186,DE102005013614). Theoretically, two illuminations are also sufficient toproduce a 3-D image. However, it is preferable for four illuminations tobe used because, in this case, the acquisitions are less sensitive todisruptions, for example, deviations in the arrangement of theilluminations or brightness tolerances.

FIGS. 2 a to 2 d show an application for photometric deflectometry withfour illuminations B1 to B4. Photometric deflectometry is a method ofoptical 3-D measurement. It represents a combination of the photometricstereo method and deflectometry. This method is suitable for reflective,glossy, and matte surfaces. For this purpose, an appropriately shapeddiffuser, for example, a hemisphere, is illuminated from variousdirections. This diffuser in turn illuminates the test specimen. Onecamera image is recorded for each illumination direction. The analysisof the various images occurs in a manner similar to the photometricstereo method. The method is described in detail in WO2004051186.

In the upper portion of each of FIGS. 2 a to 2 d, a side view of thestructure is shown. A camera K is oriented on the test specimen G in avertical manner through an opening in the diffuser S. The field ofvision of the camera K oriented on the test specimen G is implied hereby two lines L1 and L2. The circles labeled with B1 to B4 represent theilluminations. The illuminations B1 and B2 are arranged in an offsetmanner inside the image plane, which is not discernible in the sideview. The illumination currently shown as being light is the activeillumination.

In the lower portion of each of FIGS. 2 a to 2 d, a top view of thestructure may be seen. In this view, the camera K is not shown in orderto allow a view of the test specimen G. The circle that is currentlyshown as being light is the active illumination. A pixel pattern sectionA of a camera chip C has been projected onto the test specimen G inorder to clarify the synchronization of the individual illuminations B1to B4 with the camera chip C. The light diamonds represent individualpixels P of a pixel group Pn, which are activated by a shutter Sn, i.e.,connected in a light-sensitive fashion. The lower edge of the diffuser Sis also visible.

As may be seen from the lower portion of FIG. 2 a, initially theillumination B1, the pixel group P1, and the shutter S1 are active.After a predetermined exposure time, the illumination switches to B2,i.e., the illumination B1 is switched off and B2 is switched on.Correspondingly, the pixel group P2 and the shutter S2 are active, as isshown in FIG. 2 b. FIGS. 2 c and 2 d show how the cycle correspondinglycontinues with illuminations B3 and B4.

FIG. 2 e shows an application for photometric stereo. As may be seenfrom this figure, no diffuser is used in this method, and theilluminations B1 to B4 are arranged at a greater distance from the testspecimen. Otherwise, we refer to the description of the previousfigures. Again, the light illuminations, shown as circles, areactivated. For reasons of simplicity, only the active illumination B1 ofthe first cycle is shown; the other illuminations follow in an analogousmanner.

FIGS. 3 a to 3 d show the application to the stripe projection methodwith four different illumination situations that are produced by astripe projector that projects a stripe pattern onto the test specimenG. In the upper portion of the figures, a side view of the structure isagain shown. Here, a camera K is oriented in a vertical fashion on atest specimen G, and the field of view of the camera K is implied by thelines L1 and L2. Next to the camera K to its left, the stripe projectoris implied. The projection field of the stripe projector is implied hereby the lines L3 and L4. The stripe projector is arranged at an anglethat does not equal 90° to the test specimen.

In each of the lower portions of FIGS. 3 a to 3 d, a top view of thestructure is shown. The camera K and the stripe projector are not shownhere; only the field of view of the camera in the form of a pixelpattern section A of a camera chip C as well as the stripe patternproduced by the stripe projector have been partially projected onto thetest specimen G in order to clarify the synchronization of theillumination situation Bn with the camera chip C. The pixel section A ofthe camera chip C has individual pixels P. The light diamonds representthe pixels P of a pixel group Pn that is activated by a shutter Sn,i.e., connected in a light-sensitive manner.

The sinusoidal stripe patterns shown here (other patterns are possible)originate from an original pattern by phase shifting by 0°, 90°, 180°,and 270°. As is implied by the arrow 1, upon a phase shift, the patternmigrates to the left by a certain angle, i.e., the stripes of lightmove.

FIG. 3 a shows a stripe pattern in the phase position of 0° that isbeing projected onto a test specimen G. The pixel group P1 and theshutter S1 are active here. After a phase shift of 90°, the pixel groupP2 is connected in a light-sensitive manner via the shutter S2, as isshown in FIG. 3 b. The above applies correspondingly for phase shifts by180° and 270° for pixel groups P3 and P4, as is shown in FIGS. 3 c and 3d. Thus, each pixel group records a different illumination situation byvirtue of the shifting of the stripe pattern.

Stripe patterns with other stripe periods are also conceivable; agreater number of pixel groups is advisable in such a case.

Other applications, for example, to the method of interferometry, whitelight interferometry, and other methods of optical shape capture arealso conceivable.

FIG. 3 e shows the application of the method to laser interferometry.The arrangement has a laser L, a beam splitter T (semipermeable mirror),a fixed mirror P, a test specimen G, and a camera K, as well as twolenses I1 and I2. The beam emitted by the laser L having a particularwavelength is deflected by the beam splitter T on one side onto thesurface of the test specimen G and on the other side onto the mirror P.The beam is reflected by the surface of the test specimen G and itstrikes the beam splitter again. The beam striking the mirror P is alsoreflected back to the beam splitter. In the beam splitter, the two beamsmeet again and overlap. The camera K is structured in such a way thatthe overlapping light strikes it. By virtue of the different lengths ofthe paths that the light travels to the test specimen and the fixedmirror P, a difference between the light paths of 0, 1, 2, 3, 4, . . .times the wavelength of the laser, with the overlay (interference) inthe beam splitter, the brightness (constructive interference) results,and in the case of a difference of 0.5, 1.5, 2.5, . . . times thewavelength of the laser, the darkness (destructive interference)results. From the distribution of brightness and darkness (interferencestripes), the shape of the test specimen G may be suggested.

In the following, a displacement in the x direction refers to ahorizontal displacement and a displacement in the y direction refers toa vertical displacement, relative to plane of the image.

FIG. 4 a shows a schematic section of a camera chip C with four pixelgroups P1 to P4. Grey shaded areas are visible in the figure, but theseareas are for illustration purposes only and do not have any technicaleffect. One pixel P each of each pixel group Pn is located in such agrey area. In the upper left-hand grey area are the pixels P1 to P4.Pixels P1 to P4 are also provided in the upper right-hand, lowerleft-hand, and lower right-hand grey areas. A first pixel group includesall pixels P1 of all grey areas, a second pixel group includes allpixels P2 of all grey areas, and so on. All pixels P of a pixel group P1to P4 are simultaneously activated via a shutter S1 to S4. As may beseen from FIG. 4 a, the location of the four different pixel groups P1to P4 slightly from pixel group to pixel group. For example, the pixelgroup P2 is displaced by a pixel distance Δx in the x direction relativeto the pixel group P1. Correspondingly, the pixel group P3 is displacedby a pixel distance Δy in the y direction relative to the pixel group P1and the pixel group P4 is displaced by a pixel distance Δx in the xdirection and by a pixel distance Δy in the y direction relative to thepixel group P1. However, it is desirable for all four of the imagescaptured by the pixel groups P1 to P4 to be shown in a common pixelgrid. The middle point M of neighboring pixels P, i.e., pixels in a greyshaded area, lends itself to being used as a point of reference.

Alternately, another reference point may be selected, for example, thelocation of the pixels P from the group P1 or P2 etc. By interpolationof the measurement values of the pixels P from pixel group P1 alsodesignated as grey values, values are calculated using reference points,for example, using the reference point M, and in the same manner for allremaining pixel groups P2 to P4. Interpolation is also possible for adifferent number of pixel groups. For reasons of time, this calculationmay be conducted in an advantageous fashion on hardware of the cameradesigned for this purpose, as opposed to on a connected computer.

Alternately, it is also possible to provide other time patterns. FIG. 4b shows the progression over time of four shutters S1 to S4. The shuttersignals that connect the individual pixel groups Pn in a light-sensitivemanner are depicted as right-angle pulses over a certain time intervalΔt. The time axis is designated by t. For example, first the pixelgroups P1 to P4 are exposed one after the other for Δt=2 microsecondsand this process is repeated ten times until a total exposure time of 20microseconds has been attained. Thus, the temporal difference betweenexposures of consecutive pixel groups is only 2 microseconds. Theselection of times and number of repetitions represents only an exampleand may be varied virtually arbitrarily. For example, sinusoidal orotherwise modulated shutter signals may be used. The advantage ofshorter exposure times is that the test specimen has continued to move ashorter distance between the exposure of the individual pixel groups Pn.Although the total exposure time for each pixel group Pn is the same,the amount of local blurring between the pixel groups Pn caused by themoving test specimen is lower.

Alternately, a smaller or larger number of pixel groups may also beselected. FIG. 5 shows a schematic section of a camera chip C with astructure of the simplest case, namely the use of only two pixel groupsP1 and P2. Again, one pixel P each of a pixel group P1 is adjacent to apixel P of the other pixel group P2, as is implied again in FIG. 5 bythe grey shaded areas. The connection lines clarify that each pixelgroup P1 and P2 is activated by two different shutters S1 and S2. Byvirtue of the similarity to camera chips in interlaced mode (line skip),such an interlaced mode camera chip could be modified according to theinvention by a few changes. The difference from the interlaced mode liesin the fact that the even and odd lines there are exposed with a timedifference corresponding to a refresh frequency or half of a refreshfrequency. This means that the shutters that activate, for example, theodd lines, do not become active until the image acquired from the evenlines has been read. Conversely, the shutter that controls the evenlines does not become active until the image from the odd lines has beenread. The camera chip C must therefore be modified in such a way thatthe shutter time does not equal the full image period, i.e., waits forexposure and reading, but rather waits only for the exposure time and,directly after the exposure of the even or odd lines, the correspondingother shutter connects the other lines in a light-sensitive fashion.According to the invention, the time difference is therefore only theduration of the shutter time, which may easily be shorter by a factor of1000 than the total image period with exposure and reading.

Alternately, an existing interlaced camera chip C may be used incombination with a special timed control of illumination. FIG. 6 showsan exemplary progression of the shutter signals S1, S2 that connect thepixel groups P1 and P2 in a light-sensitive manner and the correspondingactivation of the illuminations B1 and B2 over time t. The shuttersignals as well as the active illuminations are depicted as right-anglepulses. The scale of the time axis t does not correspond to the scale ofthe other figures having time axes. The illumination B1, for example, isactivated for a short time shortly before the end of the shutter time ofpixel group P1. As soon as the exposure of the pixel group P2 begins,the illumination B1 is deactivated and the illumination B2 is alsoactivated for a short time. The time in which the illuminations areactivated may preferably be much shorter than corresponding to the imageperiod, i.e., the total of illumination time and reading time, forexample, from several microseconds to a few milliseconds. It is crucialhere that the different illumination situations B1 and B2 be recordedone directly after the other. Here, the pixel group P1 records the testspecimen G under the illumination situation B1 and the pixel group P2records the test specimen G under the illumination situation B2. Afterboth pixel groups P1 and P2 have been exposed one after the other, thecamera chip C may be read.

Alternately, in the case of a new camera chip design, as alreadymentioned above, a greater number of pixel groups P1 to Pn may beformed. The numbers 2×2=4, as shown in FIG. 1 a, as well as 3×3=9,4×4=16, etc. are particularly suitable because neighboring pixels P maybe combined into a square surface and the same interpolation distancesresult in the x direction and y direction, which simplifies calculationof reference points due to symmetry.

FIG. 7 shows a section of a camera chip C with 3×3 pixel groups P1 toP9. Again, neighboring pixels P of different pixel groups P1 to P9 areshaded in grey. For reasons of simplicity, the lines for shutters S1 toS9 are not shown.

Alternately, for a simpler production of the camera chip C, it may alsobe advisable not to arrange pixel groups with their own shutter signalin a square fashion, but rather in groups of 2×1, 3×1, 4×1, etc. pixels,i.e., in lines. The advantage of such an arrangement lies in the factthat each line of the camera chip C is assigned to precisely one shuttersignal and the connection layout of the camera chip C has a simplestructure. Such an arrangement for 2×1 pixel groups P1 and P2 is shownin FIG. 5. The arrangement would be expanded correspondingly for 3×1,4×1, etc. pixel groups.

An additional alternative lies in a rectangular arrangement of the pixelgroups Pn, for example, 2×3, 3×4, or other side ratios. The arrangementof eight pixel groups P1 to P8 in a 3×3 grid is also conceivable, withone grid point, for example, the middle grid point, remaining unused; areference point, for example, could be arranged there too which thesurrounding pixels P of the individual pixel groups P1 to P8 areinterpolated. In general, other arrangements, even asymmetrical ones,are conceivable as well.

The majority of the variant solutions discussed above assume thecreation of a novel camera chip design or the modification of anexisting camera chip design. The only exception is the use of aninterlaced camera chip C in which even and odd lines are already exposedat different points in time. The illumination activation must then onlybe adapted accordingly, as is shown in FIG. 6.

Additional variant solutions will be introduced below that do notrequire any changes to the camera chip design.

Method for Matrix Cameras or Line Cameras with Multiple Lines

Using a common, flat matrix camera chip that cooperates with only oneshutter S1, multiple camera images (number n) should be recorded in avery close succession, in other words, much more quickly than severalmilliseconds. Initially, this appears to be a contradiction in termsbecause such short image periods and the resulting high refresh ratecannot be achieved with a conventional method of use of a matrix camerachip.

According to the invention, the specimen to be tested is initially litwith a first illumination B1 of a total of n illuminations, the entirecamera chip C is exposed with the aid of the shutter S1, and only onepartial region of the camera chip C is read, in particular a certainnumber of lines Z1 to Zn or columns of the camera chip C or partsthereof. In the following, we will discuss only lines; however, the samealways applies to columns and parts of lines or columns. One or morelines are read, preferably as many as there are images that must becaptured with different illuminations, or a multiple thereof. In aspecial case, one or more illuminations may be dark and/or one or moreilluminations may be identical.

FIGS. 8 a to 8 d show in their top portion a side view of a structurefor photometric deflectometry. Equivalent parts have been assigned thesame reference characters; in this regard, we refer to precedingfigures. Again, the light circles represent the active illuminations,the dark circles represent the inactive illuminations. The test specimenG with a mark X, which moves in a synchronized fashion with the testspecimen G in the direction of the arrow 1, is also discernible. Thefour regions L5 to L8 designate the field of view of the camera on thetest specimen G, with each region L5 to L8 standing for one line Z1 toZ4 of the camera chip C.

The lower portion of FIGS. 8 a to 8 d shows a top view of the structure.The camera K is not shown; only a pixel grid section A of the camerachip C is partially projected onto the test specimen G in order toclarify how the camera chip C is synchronized with the illuminations B1to B4.

First, as is shown in FIG. 8 a, the illumination B1 is activated andlines Z1 to Z4 are exposed and read. A mark X on the test specimen G,which has been selected by way of example, is in motion at this point intime in the region of the line Z4. Then, a shift occurs to a secondillumination B2, and another partial region of the camera chip C isread, preferably the same region that recorded the previous illuminationsituation with B1. The test specimen G, and with it the mark X, has inthe meantime moved farther by a certain distance. The mark x is now, forexample, located in the region of the line Z3, as is shown in FIG. 8 b.

This sequence repeats, as is shown in FIGS. 8 c to 8 d, until a numberof illuminations n has been achieved. Then the next series of nilluminations and reading processes follows until a desired number ofrepetitions has been reached. Here, the test specimen G is in motionrelative to the camera, such that the surface of the specimen or partsof it are covered over. This motion may occur in various manners, forexample, motion in a straight line at a constant speed and in a constantdirection. For rotationally symmetrical specimens whose coatings are tobe tested, a rotation around their symmetrical axis at a constantrotational speed recommends itself.

Another variant, which is not shown here, is a movement of the testspecimen G in which, in addition to rotation, a simultaneous feeding ofthe test specimen G along the rotational axis occurs. Thus, one point onthe surface of the test specimen G described a helical path. Here, thelines of the camera chip C must be configured in such a way that theyare situated perpendicular to the helical line. One advantage of thismovement is that even long cylindrical components may be completelycaptured within several rotations. However, multiple individual testspecimens may also be arranged one after the other and, in this manner,continually tested.

FIG. 8 e shows by way of example the structure for the photometricdeflectometry method; however, in contrast to FIGS. 8 a to 8 d, thisfigure shows a test specimen G that is rotating in the direction of thearrow 1. Again, cones L5 to L8 imply the field of view of the cameraand/or the recording locations of the individual lines Z1 to Z4 of thecamera chip C on the test specimen G.

From capture to capture, the test specimen G continues to move along acertain route or by a certain rotational angle. The speed, rotationalspeed, time of capture, and/or time of reading are advantageouslyselected such that the test specimen G moves from capture to capturerelative to the camera by a pixel distance or a whole-number multiplethereof. This may be achieved, for example, by an encoder coupled withthe motion, in particular a rotational encoder, that specifies the imagecapture time point and reading time point correspondingly such thattolerances of the motor do not have any negative effects.

In the upper portion of FIG. 9 a, the location of the test specimen G,which is being captured by lines Z1 to Z4, is shown (vertical axis) as afunction of time (horizontal axis).

The lower portion of FIG. 9 a shows which illumination B1 to B4 isactive. The illuminations are activated one after the other; at the sametime, lines Z1 to Z4 are connected in a light-sensitive manner. Aftereach illumination process, lines Z1 to Z4 are read.

Over time, lines Z1 to Zn gradually capture various locations withregard to the test specimen G. The number of lines and the number ofilluminations were selected by way of example at n=4. Other numbers nare possible as well.

The image areas that have been read, in this case lines Z1 to Z4, aretransmitted to an arithmetic unit R and stored in a memory there, withsaid storage occurring according to a certain pattern.

FIG. 9 b shows a schematic depiction of such a memory pattern that isarranged in a memory area depicted in a highly abstract manner. Theindividual boxes Z1 to Z4 represent the measurement values that thelines Z1 to Z4 of the camera chip C recorded from the respective surfacesection of the test specimen G with a certain illumination situation.The storage order of the measurement values is organized byilluminations. First, therefore, lines Z1 to Z4 are illuminated with B1and stored in the memory, then lines Z1 to Z4 are exposed withillumination B2 and stored one line below in the memory. This processrepeats for all four illuminations. Then the storage process beginsagain subsequent to the first storage. Here, in each grey shaded memoryregion, an image results that was recorded with illuminations B1, B2,B3, and B4, with the recorded surface of the test specimen G running inthe direction of the arrow O. In the figure, the respective illuminationB1 to B4 with which the test specimen was illuminated is given undereach column. Lines Z1 to Z4 are stored in the memory in such a way thatthe same mark X on the surface of the test specimen G is arranged in ahorizontal row. Here, so-called corresponding lines result thatcorrespond to the same location on the surface of the test specimen withdifferent illumination situations. One example of corresponding lines inthe case of illuminations B1 to B4 is designated with K in the figure.In the case of this example, “corresponding” means that the line Z4under illumination B1 shows the same region of the test specimen G, forexample, the mark X, as line Z3 under illumination B2, Z2 under B3, andZ1 under B4. Thus, pixel-identical data, i.e., data of the same surfaceregion, are present for each illumination B1 to B4. This is a particularadvantage because it eliminates the need for interpolation betweenintermediate values. This leads to less laborious calculations and lessexpenditure of time. Moreover, errors that would otherwise occur duringinterpolation can be avoided. Such errors particularly occur in the caseof sharp brightness transitions in the image, area that are often ofgreat interest.

The lines that were recorded with different illuminations are arrangedone next to the other horizontally in FIG. 9 b. This two-dimensionaldepiction is for the sake of clarity, so that corresponding lines may beshown in the most comprehensible manner possible. Naturally, thesememory regions in the storage medium may also be arranged in analternate manner. For example, the arrangement of the lines in thememory may also have a linear concept. In the simplest case, the linesof the camera are stored one directly after the other in the memory in alinear fashion and corresponding lines are assigned with reference totheir distance from one another in the memory. All of these differentarrangements in the memory should be viewed as equivalent to the memoryarrangement shown, as long as it is possible to designate correspondingmemory regions. This also applies to subsequent figures that show thememory.

As an alternative to FIGS. 9 a and 9 b, the displacement may also be amultiple of the pixel distance, for example, twice the pixel distance.In this case, eight lines each must be read. The arrangement in thememory of the arithmetic unit is then preferably selected as in FIG. 10a, which also shows a schematic depiction of a memory pattern. Again,the individual boxes Z1 to Z8 represent the information that the linesZ1 to Z8 of the camera chip C have recorded of the respective surfacesection of the test specimen G with a particular illumination situation.The recorded surface of the test specimen G runs in the direction of thearrow O.

The reading times achieved in this method are typically lower by afactor of 100 to 1000 than reading the entire matrix camera chip byvirtue of the fact that only partial regions of the matrix camera chipare read. The factor of 100 results in cases in which only 4 lines areread instead of 400; the factor of 1000 results when only 2 lines areread instead of 2000. For this reason, the variant for n=2 is ofparticular interest.

The upper portion of FIG. 10 b shows the location on the surface of thetest specimen G (vertical axis) that the two lines Z1 and Z2 of thecamera chip C record as a function of time (horizontal axis).

The lower portion of FIG. 10 b shows illuminations B1 and B2, depictedas right-angle pulses, which are activated in an alternating fashion.The test specimen continues to move, as may be seen from the upperportion of FIG. 10 b, by a line distance of the camera chip C, lines Z1and Z2 are again connected such that they may be exposed, and the secondillumination B2 is activated.

FIG. 10 c shows the corresponding memory pattern in a highly schematicmemory when two lines Z1 and Z2 are used. Again, information is shownregarding the surface of the test specimen G in the boxes Z1 and Z2. Thesurface of the test specimen G again moves in the direction of the arrowO. A corresponding line K clarifies that the same surface regions of thetest specimen G that were recorded with different illuminations B1 andB2 are arranged one next to the other in a horizontal row.

Additional exemplary applications are shown in FIGS. 11 a to 14 b. FIG.11 a shows an application of the method to the photometric stereo methodand a linear movement of the test specimen G, which moves in thedirection of the arrow 1. In contrast to photometric deflectometry, thediffuser has been eliminated here and the illuminations B1 to B4 arearranged at a greater distance from the test specimen G. In the upperportion of FIG. 11 a, the field of view of the camera K is designated byregions L5 to L8, which represent individual lines of the camera chip C.The same parts are designated using the same reference characters; inthis regard, we refer to the preceding figures.

FIG. 11 b shows the method applied to the photometric stereo method incombination with a rotational movement by the test specimen G, whichrotates around the rotational axis 3 in the direction of the arrow 1.

In FIG. 12 a, the method according to the invention is applied to thestripe projection method, which is known per se, and a linear movementby the test specimen G.

In the upper region of FIG. 12 a, a side view of the structure may beseen. The field of view of the camera K is designated by regions L5 toL8, which represent individual lines of the camera chip C. A stripeprojector SP illuminates a region of the test specimen G with a stripepattern whose area is designated by the lines L3 and L4. Preferably, thestripe projector provides illumination with a sinusoidal modulation ofillumination intensity depending on the location on G. The test specimenG preferably moves in a linear and even fashion in the direction of thearrow 1.

In the lower portion of FIG. 12 a, a top view of the structure may beshown. Here, the test specimen G, the strip pattern projected thereon,and a projection of the field of view of lines Z1 to Z4 of the camerachip C are shown.

FIG. 12 a shows how a mark X is initially located in the region of theline Z4, then in the region of the line Z3 according to FIG. 12 b, thenin region of the line Z2, and in the region of the line Z1, as is shownin FIGS. 12 c and 12 d.

In the case of stripe projection, the stripe pattern may be varied fromtime segment to time segment in its phase position and stripe distance(this corresponds to the different illuminations B1 to Bn); however,this is advantageously not necessary. The movement of the test specimenalone ensures that a mark X will be located in different regions of thestripe pattern at different points in time (in different phase positionsof the stripe pattern). Advantageously, the stripe distance is selectedsuch a way that, after n movement steps, precisely one stripe period,i.e., one light stripe and one dark stripe, will be crossed. In thepresent example, n=4. Thus, the mark X is recorded in line Z4 with aphase position of 0° (B1), in Z3 with a phase position of 90° (B2), inZ2 with 180° (B3), and Z1 with 270° (B4). This motion thereforeautomatically creates an appropriate phase shift that must first belaboriously produced in other methods.

The particular advantage of the method according to the inventiontherefore lies in the fact that an extremely simple and cost-effectiveprojector may be used with a static, unchanging stripe pattern and ahigh degree of stripe contrast. In this case, the temporal succession ofthe n illuminations is replaced by the spatial progression of nilluminations. Each phase position of the illumination, for example, 0°,90°, 180°, 270°, which occur one next to the other spatially,corresponds to an illumination Bn. It is possible for one or moreadditional such projectors with a different stripe distance to be usedin order to eliminate ambiguity between various stripes. This ambiguityoriginates from the fact that the strip pattern is sinusoidal; dark andlight strips repeat periodically. These projectors may also each projectone single pattern and are activated one after the other. If the stripeperiod is greater than n adjacent lines, n lines may also be read at arespective distance by more than one pixel. A further advantage of themethod according to the invention lies in the fact that there is anextremely short period of time between image acquisitions in variousphase positions, and interference with measurement due to vibrations maybe suppressed. Thus, by virtue of the fact that the image acquisitionduration is very brief, the vibration of the arrangement does not have anegative effect on the image acquisition; therefore, vibrationstabilization of the measurement structure is not necessary.

FIG. 12 e shows the method according to the invention for application instripe projection and a rotational movement by the test specimen G inthe direction of the arrow 1 around the rotational axis 3. The sameparts are designated using the same reference characters; in thisregard, we refer to the preceding figures.

FIG. 13 a shows the method according to the invention applied to theinterferometric testing method, which is known per se, and isparticularly suitable for optically smooth surfaces because it ispossible to detect very fine structures whose roughness is less than thewavelength of visible light, i.e., less than approximately 0.5 μm.

In the upper portion of FIG. 13 a, a temporally coherent light source,for example, a laser L, may be seen, which serves as the light source.The field of view of a camera K is directed at the test specimen G.Regions L5 to L8 represent the acquisition regions of lines Z1 to Z4 ofthe camera chip C. A mark X is located in the region of line Z4. Thetest specimen is moving in the direction of the arrow 1. The beamemitting from the laser L with a certain wavelength is deflected on oneside by a beam splitter T onto the surface of the test specimen G and onthe other side onto a mirror P. On the surface of the test specimen G,the beam is reflected and returns to the beam splitter T. The beamstriking the mirror P is also reflected back to the beam splitter T. Inthe beam splitter T, the two beams meet again and overlap. The camera Kis structured in such a way that the overlapping light strikes it. Byvirtue of the different lengths of the paths that the light travels tothe test specimen G and the fixed mirror P, a difference between thelight paths of 0, 1, 2, 3, 4, . . . times the wavelength of the laser,with the overlay (interference) in the beam splitter T, the brightness(constructive interference) results, and in the case of a difference of0.5, 1.5, 2.5, . . . times the wavelength of the laser, the darkness(destructive interference) results. From the distribution of brightnessand darkness (interference stripes), the 3-D shape of the test specimenG may be suggested.

In the lower portion of FIG. 13 a, a top view of the test specimen G aswell as projections of lines Z1 to Z4 of the camera chip C and theinterference pattern created thereby may be seen. The mark X on thesurface of the test specimen may also be seen; here, it is located inthe region of line Z1.

As described above, a Michelson interferometer is shown by way ofexample as the interferometer type; however, other types such asMach-Zehnder and many others are possible as well. Although themechanism of producing the interference stripes is completely differencefrom that of stripe projection, the method according to the inventionmay be transferred from stripe projection to interferometry in avirtually 1:1 fashion. Here as well, by suitably selecting the stripeperiods (for example, by tilting the mirror P or the plane of the testspecimen G) to fit the distance of n lines, it is possible to be able tocapture phase-shifted intensity values in the n lines; for n=4, 0° inline Z4, 90° in Z3, 180° in Z2, and 270° in Z1. Here as well, asignificant simplification and reduction in costs results because astatic interferometer may be used without a device for phase shifting.In this case, the temporal succession of the n illuminations is replacedby the spatial progression of n illuminations. Each phase position ofthe illumination, for example, 0°, 90°, 180°, 270°, which occur one nextto the other spatially, corresponds to an illumination Bn. Sensitivityto vibrations of the measuring apparatus and/or the interferometer isalso significantly reduced. By virtue of the fact that the imageacquisition occurs so quickly, vibrations that are caused byenvironmental influences do not have any negative effects on imageacquisition.

Alternately, different stripe distances may be realized, for example, byone or more additional light sources with different wavelengths(multiple-wavelength interferometry). These light sources may beconnected with a short time difference from one another, in particularwhen they are semiconductor lasers.

FIG. 13 b shows an application for rotational movement of a testspecimen G in the direction of the arrow 1 around a rotational axis 3and for the capture of essentially rotationally symmetrical testspecimens G which up to now were accessible for interferometric testingnot at all or only with difficulty. The same parts are designated usingthe same reference characters; in this regard, we refer to the precedingfigures.

FIG. 14 a shows the method according to the invention for the method ofwhite light interferometry, which is known per se and which is suitablefor optically smooth surfaces as well as for optically rough ones, i.e.,surfaces whose roughness is greater than approximately 0.5 μm.

In the upper portion of FIG. 14, a light source LQ, a camera K, a beamsplitter T, and a mirror P are shown. The field of view of the camera Kis directed at the test specimen G, which is moving in the direction ofthe arrow 1. The light source LQ may, for example, be LEDs, incandescentlamps, or the like. Multiple wavelengths contribute to the incidence ofinterference. The maximum brightness results when both light paths,i.e., from the beam splitter T to the mirror P and back, and from thebeam splitter T to the test specimen G, have the same length. As a rule,it is necessary to displace either the mirror P or the test specimen Gin very many small steps (for example, by one quarter of a wavelengtheach time) in order to equalize the two light paths. Variations inbrightness are then captured by the camera K.

In this method, it is advantageous for the stripe periods to be selectedin such a way that, for example, four lines Z1 to Z4 correspond to astripe period; other numbers are possible as well. Because the stripecontrast in white light interferometry increases or decreases from onestripe to the next, it is advisable here to record more than one stripeand, for example, to read 100×4 lines. The maximum for the stripecontrast indicates the height above the plane of movement at which amark is located. Here as well, advantages of the method according to theinvention are that no device is required for phase shifting and thestructure may therefore be realized in a simple and cost-effectivemanner, and influences of vibrations may be suppressed by the rapidimage capture.

Here as well, the temporal succession of the n illuminations is replacedby the spatial progression of n illuminations. Each phase position ofthe illumination, which occur one next to the other spatially,corresponds to an illumination Bn. In addition, the white lightinterferometry is practicable for measuring the coating surface ofessentially rotationally symmetrical parts. FIG. 14 b shows a structurein a side view and a top view in which a rotationally symmetrical testspecimen G rotates in the direction of the arrow 1 around a rotationalaxis 3.

Alternately, instead of a matrix camera, a special camera K may be usedas well whose camera chip C includes only a few lines. Of particularinterest are cameras and chips that, for example, include two, three,four up to approximately 100 lines. Such cameras represent thetransition between line cameras and matrix cameras.

Such cameras with n=3 lines are available as color cameras. Here, everyline is provided with its own color filter. Black and white cameras withthree lines, which are also available, are even better suited for themethod according to the invention. The special feature of such camerascan lie in the fact that the distance between neighboring lines is amultiple of the pixel distance within a line, for example, eight times.The test specimen may continue to move by the line distance from captureto capture, then the method according to the invention is to beperformed as described. Advantageously, however, the test specimencontinues to be moved only by one pixel distance from acquisition toacquisition. In this case, it is advantageous for n=3 illuminations tobe actively connected for 8 captures before switching to the nextillumination. This process is equivalent to 8n=24 lines being read inthe case of a chip with a line distance corresponding to the pixeldistance. The arrangement of the images in the memory may then occuradvantageously analogously to FIG. 10 a and 24 lines. Alternately, it isalso possible for only n=2 of the three camera lines to be used. Then itis advantageous for the arrangement in the memory to correspond to FIG.10 a and 16 lines.

Additional cameras with a higher number of lines are available thatinclude, for example, 96 or 100 lines. Such cameras are frequently usedin accordance with the TDI (time delay integration) method. However manyof these cameras may also be used as matrix cameras (area mode). Thesespecial cameras may be advantageously used for the method according tothe invention, in particular when they are operated as matrix cameras.As a rule, the line distance here is equal or similar to the pixeldistance that is common for other surface cameras.

Method for Multiple Matrix Cameras

Moreover, additional variants are conceivable in which standard matrixcameras with only one shutter S each are used and yet a short sequenceof images over time is attained. For this purpose, a stereo arrangementof n cameras K1 to Kn is modified in such a way that they are able tofulfill this purpose. Here, the camera chips C of the cameras Kn are notexposed at the same time as is generally common, but rather with aslight temporal offset. According to the invention, this temporal offsetis much shorter than would correspond to the refresh rate. This offsetadvantageously corresponds to the exposure duration of a camera chip C.The n illuminations B1 to Bn are controlled in such a way that a firstillumination B1 occurs simultaneously with the exposure of the camera K1with the shutter S1, the second illumination B2 with the exposure of thecamera K2 with the shutter S2, up to the illumination Bn of the cameraKn with the shutter Sn. Alternately, it is possible for the illuminationto be activated for less time than corresponds to the shutter time. Hereas well, n=2 is a particularly interesting variant.

Directly after the exposure of the camera chip C of a camera Kn by anillumination Bn, the camera chip C of a next camera Kn is exposed with anext illumination Bn. This process repeats itself. Only after theexposure of a camera chip C of the camera Kn, a part of the camera chipC is read. Corresponding lines, i.e., lines of the various camera chipsC that were acquired under various illumination situations, are arrangedcorrespondingly in a memory taking into account of a disparity of thecamera Kn.

Initially, this approach appears absurd because, in a stereoarrangement, an offset of the images from camera K1 to Kn exists,usually intentionally. From this offset, known as a disparity, themeasurement signal is commonly attained, namely the spatial depth. Everyvariation of the disparity means a height difference of the object to bemeasured and is determined as precisely as possible. For the methodaccording to the invention, the opposite is the case; here, thedisparity is minimized as well as possible and kept constant. This isattained by virtue of the fact that the cameras are placed at as small adistance as possible from one another.

FIG. 15 describes a structure with two cameras K1 and K2. Preferably,both cameras are oriented with their optical axes parallel to oneanother. The test specimen G is positioned in a plane perpendicular tothe optical axes of the cameras K1 and K2. It is particularlyadvantageous for the test specimen G to be guided always in this planein an essentially even fashion and by a—preferably mechanical—device.

The disparity is then merely a constant offset by a certain number ofpixel distances in all areas of the images. Preferably, the distance ofthe cameras is adjusted precisely in such a way that the disparity is awhole-number multiple of the pixel distance. Thus, it is possible toacquire corresponding pixels without interpolation. Alternately, thedistance may be freely selected or the orientation of the camera axesmay even be freely selected. In such cases, however, an interpolation isnecessary in order to acquire corresponding pixels, i.e., pixels of thesame location on surface of the test specimen.

If more than two cameras are used, they may be arranged in a row or evenin a planar fashion. For example, four cameras may be placed in a square2×2 arrangement, which minimizes distances. The special case for adisparity of zero may be achieved if two cameras are oriented inprecisely the same manner on the test specimen G via a beam splitter.

Method for One Camera and Multiple Rotations

All of these variant solutions have in common that a series of imagesmay be recorded with the smallest time delay, in particular when theobject to be tested is in motion. For the particular case in which thetest specimen G is rotating, for example, when the coating surface of arotationally symmetrical test specimen should be captured, an additionalalternative is possible: here, the test specimen G is scanned during itsrotational movement by a line Z1 of a camera chip C while a firstillumination B1 is active.

FIGS. 16 a to 16 d show an application of the method according to theinvention with photometric deflectometry by way of example with fourilluminations B1 to B4. The same parts are designated using the samereference characters; in this regard, we refer to the preceding figures.

In the upper portion of FIGS. 16 a to 16 d, a side view of the structurefor photometric deflectometry is shown. The region L9 in the upperportion of FIGS. 16 a to 16 d designates the field of view of the lineZ1 directed onto the test specimen G. The test specimen G rotates in thedirection of the arrow 1 around the rotational axis 3. Moreover, a markX is located on the rotating test specimen G. The respective lightcircles designate the active illumination.

The lower portion of FIGS. 16 a to 16 d depicts a top view of thestructure without the camera K. A surface segment of the test specimen Gis discernible onto which a line Z1 of the camera chip C of the camera Khas been schematically projected, such that the synchronization betweenthe illuminations B1 to B4 and the line Z1 may be shown.

Other methods such as, for example, the photometric stereo method,stripe projection, and interferometric methods are also possible. As isshown in FIG. 16 b, after a full rotation, the illumination B2 isactivated instead of the illumination B1. After another rotation, theswitch is made to the illumination B3 and, after yet another rotation,to the illumination B4, as is shown in FIGS. 16 c and 16 d. The desirednumber of illumination situations may vary. Here, it is important that,for every rotation, the individual lines be captured in a preciselycongruent manner to the prior rotations. This may be achieved, forexample, using a shaft encoder.

It is particularly advantageous if a line camera, but in particular ahighly light-sensitive light camera, is used for the method according tothe invention. Such a highly light-sensitive line camera, which is knownper se, follows a mark X on the test specimen G over multiple lines bycharge displacement and thus achieves a summation effect in exposure.This technique is known as TDI (time delay integration) technology.

FIG. 16 e shows the method according to the invention along with such acamera. In the lower portion of FIG. 16 e, a line pattern of the camerachip C of the camera K is projected onto the test specimen G. Theaccumulation of loading from right to left is implied by an increasinggrey shading. A particular advantage of this embodiment of the inventionlies in the fact that the normally low amount of light in the case ofline camera applications may typically be increased by a factor of 100and, at the same time, the exposure time is reduced and a more rapidmovement of the test specimen is made possible. In this manner, adrastic reduction in the capture and inspection time as well as in thecapture and inspection costs is achieved.

The invention claimed is:
 1. A method for recording a number of imagesof a test specimen in rotational motion with a camera using n differentilluminations (B1 to Bn), the method comprising: activating theillumination for at least each full rotation of the rotating testspecimen; and capturing with the camera the same point on the testspecimen to be inspected for at least n complete rotations of the testspecimen, wherein n>1.
 2. The method according to claim 1, furthercomprising using a highly light-sensitive line camera.
 3. The methodaccording to claim 2, used in conjunction with a photometricdeflectometry method.
 4. The method according to claim 2, used inconjunction with a photometric stereo method.
 5. The method according toclaim 2, used in conjunction with a stripe projection method.
 6. Themethod according to claim 2, used in conjunction with an interferometrymethod.
 7. The method according to claim 2, used in conjunction with awhite light interferometry method.
 8. The method according to claim 2,used in conjunction with a stereo method.